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ABSTRACT 


The terminal environment in the aviation domain is referred to as the region 
within close proximity of airports. Performance in this environment leaves little room for 
error and requires that pilots are fully aware of their situation and surroundings. This is 
especially true when conducting an instrument approach (lA). Currently, United States 
Marine Corps (USMC) KC-130J units do an exceptional job of training their pilots in this 
environment. However, the emergence of new technologies affords the opportunity to 
address shortfalls of legacy training approaches. To this end, this thesis uses a quasi- 
instructional systems design approach to develop a prototype lA part-task trainer (PTT) 
using commercial off-the shelf (COTS) systems to enhance current training methods to 
allow for training opportunities that are not limited by schedules or location. After 
the development of the PTT, subject-matter experts evaluated the prototype training 
system. Their feedback suggests that the system reliably re-created necessary cues 
(cockpit displays, environmental conditions, communications, time, and space) to build 
on users’ existing knowledge and support situation awareness training for lA. 
Ultimately, the proof-of-concept prototype created for this thesis is envisioned to be 
part of a family of solutions capable of addressing existing training gaps in a cost- 
effective manner. 
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I. INTRODUCTION 

A. RESEARCH DOMAIN 

An instrument approach (lA) is one method used to successfully land an aircraft 
at an airport. When an lA is being flown, there are several basic procedures that the pilot 
flying (PF) must accomplish in order to successfully land that aircraft. These basic 
procedures typically follow a sequence that does not change from approach to approach, 
so the aviator becomes very good at accomplishing this sequence successfully. However, 
when an emergency situation occurs during the flight and an lA must be flown to land the 
aircraft, these basic procedures could become difficult to accomplish. These emergency 
situations often require heighten situation awareness (SA) so that the basic procedures 
can be accomplished while also successfully completing the procedures of the 
emergency. Having sound SA is crucial in these situations for one major reason: if all 
basic and emergency procedures are not successfully completed during the lA, a 
catastrophic event could occur. 

United States Marine Corps (USMC) KC-130J replacement pilots (RP) receive 
initial lA training at a Fleet Replacement Detachment (FRD). At the FRD, a RP conducts 
ground school, which consists of self-paced computer-based training (CBT) and 
instructor-led discussions focused on studying the missions of the KC-130J. In addition, 
there is material within different publications that a RP needs to know. The instrument 
approach procedure (lAP) elements of ground school consist of generic lAP for all 
aviators and lAP specific to KC-130J aviators. The RP will then begin to fly lAs in the 
KC-130J simulator. Initially, these lAs consist of honing these basic procedures so that 
the RP could acquire a solid level of confidence while executing those procedures. In 
these early stages of training at the FRD, it is commonly accepted that it is more 
important that the RP polishes these basic procedures rather than successfully fly the 
simulator. Successful flight of the simulator will come with practice, and it will be 
enhanced with mastery of the basic lAPs. 
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B. RESEARCH PROBLEM AND MOTIVATION 


When emergency procedures (EP) are introduced to an lA, the aviators, especially 
RPs, could experience some difficulties. Repetition of basic procedures is extremely 
important to a RP; the repetitive training approach can build the required foundation of 
these procedures so that they become second nature. However, aviators cannot use the 
simulator and aircraft to groom and acquire these procedures at their leisure. The 
simulator and aircraft follow a very rigid schedule, and the aviator may only have access 
to the simulator and aircraft two times a week. Whereas the simulator and aircraft provide 
potentially better opportunity to properly practice the procedures, their availability does 
not allow for the consistent repetition that is necessary. 

Another technique called chair flying is a non-computer based technique, and 
aviators use it to practice any procedure on their own; this is typically done without the 
supervision of instructors. This approach has modest demand on resources—chair flying 
does not require anything but a chair. Though, chair flying is only useful if the aviator is 
practicing all procedures properly. This is especially true of a RP because at that stage in 
their career and training the RP may not have the ability to recognize that he or she is 
practicing the procedure incorrectly. It is worth emphasizing that negative practice could 
transfer to negative execution of procedures within the simulator and aircraft. 

The instructor-led discussions are the only portion of the classroom environment 
that is required of the student. The knowledge of the computer-based training and 
readings are upon the student. However, the RP is required to combine the knowledge 
from these different avenues and apply it mentally and physically within the simulator 
and aircraft. The act of combining the required mental knowledge with the required 
physical actions of operating the cockpit while flying the aircraft can be overwhelming to 
the RP. 

Proper repetitive practice of any procedure is essential to the development of this 
procedure. Since simulators and aircraft are not readily available and chair flying does 
not ensure proper practice, the development of a PTT could address that gap. If such a 
PTT would use a COTS platform that is inexpensive and easy to acquire, such as a tablet. 
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the PTT would afford the aviator the cost-effective opportunity to practice procedures 
anywhere at any time. General Robert B. Neller (2016), the Commandant of the Marine 
Corps (CMC), has stated that the Marine Corps can be “enabled by technology, we will 
increase the amount of training each unit can accomplish—to ‘increase the reps’ in 
mentally and physically stressing environments for all elements of the MAGTF before 
they do so on the battlefield” (p. 8). With the guidance of the CMC, the development of 
such a PTT becomes imperative to the Marine Corps future training programs. This type 
of affordable yet highly capable devices has a great potential of bridging a gap that 
currently exists in the training pipeline of any community in the Marine Corps. 

C. RESEARCH QUESTIONS 

The foundation of the thesis is based on the following research questions: 

• What is the feasibility of developing a prototype virtual reality (VR) 
system that supports the training of instrument approach procedures and 
situation awareness? 

• Are there training gaps identified in connection with procedures and 
situation awareness needed to conduct an instrument approach by the 
aviators? 

• What cues (cockpit, environmental, CRM, ATC, haptic, etc.) should KC- 
130J FRD students attend to in support of maintaining situation awareness 
during instrument approaches? 

D. SCOPE 

The scope of this thesis is to investigate current techniques used by Marine Corps 
KC-130J squadrons and the FRDs to train aviators for lAP. A PTT will be developed as a 
proof-of-concept prototype with a goal to become a supplemental tool to the current 
training that aviators receive. A significant portion of this work will address the aspects 
of SA and how SA relates to lAP. The study also includes gathering of informal feedback 
during demo sessions. A training effectiveness study is not part of this thesis, yet it is 
envisaged as elements of follow-up work. 
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E. APPROACH 


The approach used in support of the thesis research has consisted of analyzing 
literature related to SA and how a PTT enhances SA so that the basic lAP are committed 
to memory and conducted without thought. In addition, a user survey is conducted to 
provide a better understanding of preferences, habits, and recommendations from the 
fleet. The research of SA and the survey results were then used to design and develop the 
PTT. A demo of the prototype system was given to fellow aviators, and their informal 
feedback was acquired to help understand the value of the developed PTT. 

F. METHODOLOGY 

1. Introduction 

The basis for this thesis is the work presented in a past NPS thesis involving 
Marine aviation (Attig, 2016). This thesis is not continuation work of those results but 
rather an effort to apply the basic concepts presented there to a different domain of 
Marine aviation—lAPs—and provides a novel PTT that could become a valuable tool to 
the Marine Corps’ future training environment. Initially, multiple missions of the KC- 
1301 community were suggested, and after much deliberation, the lAPs were chosen 
because they are among the first missions that RPs must learn. The PTT that resulted 
from this thesis effort gives the aviator the ability to run the scenario and observe the 
proper procedures associated with an lAP as it is conducted in the KC-130J. 

2. Design Process 

To begin with, a literature review of situation awareness, knowledge types, and 
PTT was conducted. These issues are the basis of this thesis, and it is imperative to 
express how the information acquired about each topic applies to the idea of using 
technology to assist with future training aspirations of the Marine Corps. The task 
analysis for the PTT was developed from the KC-130J Super Hercules’ Naval Air 
Training and Operating Procedures Standardization (NATOPS) documentation. Within 
the NATOPS, sections of chapters 7 and 16 were used to generate the elements of task 
analysis (Chapter 7 was focused on the in-range, approach, and before landing flows and 
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checklists material, and Chapter 16 addressed the standard callouts required while 
executing an lA) (Department of the Navy, 2014). In order to assist with the design of the 
PTT, two surveys were conducted to collect valuable information from current IPs and 
RPs. Not only would these answers provide valued information for this thesis’ success 
but the information was also used to inform the thoughts for follow-on work. Prior to 
developing these surveys, subject matter experts were questioned to assist with the 
development of the survey questions from the current atmosphere across the KC-130J 
community. 

G. THESIS STRUCTURE 

The remainder of this thesis is structured as follows: 

• Chapter II describes SA, COTS, and knowledge. 

• Chapter III details task analysis focused on instrument approach. 

• Chapter IV presents the results of the instructor pilot (IP) and RP surveys 
and describes the process of developing a prototype trainer. 

• Chapter V describes the informal feedback from fellow aviators and 
includes a summary of conclusions and future works related to this study. 
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II. BACKGROUND 


A. INTRODUCTION 

The terminal environment of aviation is very dynamic and requires an aviator to 
maintain an elevated level of SA. It could also be argued that aviators must maintain a 
higher level of SA than air traffic control and ground personnel in this environment. Even 
under the most ideal conditions, this environment can become overwhelming for even the 
most experienced aviators. A less than appropriate level of SA could lead to disastrous 
results for many of the individuals involved. SA is comprised of many factors, and one 
factor is often dependent upon another factor or factors. 

B. SITUATION AWARENESS 

Situation Awareness has been widely studied by researcher Mica Endsley; 
Endsley (1995) describes SA as having three separate levels. Her definition of SA is “the 
perception of the elements in the environment within a volume of time and space, the 
comprehension of their meaning, and the projection of their status in the near future” (p. 
36). 

The first level of SA is the ability to perceive the pieces of information, objects, 
etc. Perception is related to an individual’s knowledge and can be influenced by the 
design of the system that supports SA (Endsley, 1995, pp. 36-37). She notes that if an 
individual has the necessary knowledge and the system is designed to take advantage of 
human information processing, the likelihood that the salient cues will be perceived is 
high (pp. 36-37). Initially, an individual’s knowledge needs to be at a level that is 
suitable to fly an lA within the terminal environment. An aviator’s knowledge does not 
have to be at an expert level, but he or she needs to know all the vital information needed 
to safely and successfully perform an lA. This includes the declarative and procedural 
knowledge concerning where and how to read an approach, frequencies, navigational aids 
(NAVAID), approach course, altitude, course adjustments, etc. Additionally, an aviator 
needs to know the Eederal Aviation Administration’s guidelines to flying an lA such as 
clearance procedures and missed approach procedures. If a system is poorly designed, it 
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is likely that an aviator’s knowledge alone will not be sufficient enough to achieve or 
maintain a high level of SA. Hence, a well-designed system that enables the perception of 
environmental cues combined with the requisite knowledge will support an individual in 
successfully achieving level 1 SA and have a foundation for the higher levels of SA. 

Level 2 SA is the comprehension of the elements within the current situation. 
Comprehension signifies one’s ability to take the perceived information from level 1 and 
piece that information together in order to create a complete mental picture of the world 
in context of the operating environment (Endsley, 1995, p. 37). Simply put, level 2 SA is 
the ability to understand the current state of the perceived elements in the environment. 
Being able to create this mental picture is a crucial aspect of developing and maintaining 
a high level of SA within any given situation. 

What is considered the highest level of SA, level 3 SA, is one’s ability to 
accurately simulate the future activities of the elements perceived within the context of 
the individual’s level 2 SA of the world (Endsley, 1995, p. 37). Beyond perceiving and 
comprehending relevant information, Eevel 3 SA is the convergence of “comprehending 
the meaning of that information in an integrated form, comparing it with operator goals, 
and providing projected future states of the environment that are valuable for decision 
making” (p. 37). 

Endsley and Garland (2000) state that the proposed levels of SA describe a model 
for acquiring information, processing it, and applying it accurately. They claimed an 
individual’s goals are what determine what would be considered appropriate SA. The two 
reported that if goals change, what comprises appropriate SA might need to be adjusted 
just as well. If maximal performance is desired, an individual must strive to have the 
highest possible SA in order to support the optimal decisions making on the associated 
tasks, their study showed. However, desired outcome expectations can have an impact on 
these goals. An individual may want to perceive information in a different manner than is 
required because they were expecting something different according to Endsley and 
Garland. According to the authors, it is preferable to have appropriate expectations so 
that when anticipated information is perceived it can be processed effectively. However, 

it is more important to have the ability to recognize that the information is not as 
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expected and the ability to process it properly per their study. Complexity of a situation 
can be related to the goal. In Endsley’s 1995 work, she states that if the goal is too 
difficult, the complexity of a situation could become overwhelming and detract from an 
individual’s ability to maintain sound SA (Endsley, 1995, p. 53). This is especially 
relevant for situations in which an overall goal is comprised of numerous sub goals. 

SA is the main precursor to decision-making. Yet, many factors occur between 
SA and good decision-making (Endsley & Garland, 2000, pp. 4-5). An individual may 
have a high SA, but their performance may be poor due to circumstances outside of their 
control such as a faulty user interface or lack of information from third parties. In 
addition, the information available to an individual as they developed their three levels of 
SA may be insufficient, and the individual’s perception of the situation can become 
significantly subpar. The key takeaway is that “SA, decision-making performance can be 
seen for theoretical purposes to be distinct stages that can each affect the other in a 
circular ongoing cycle, yet which can be decoupled through various other factors” (p. 5). 

The measurement of SA is meaningful because it “provides a useful index for 
evaluating system design and training techniques and for better understanding human 
cognition” (Endsley & Garland, 2000, p. 17). This notion is essential because if the goal 
is to achieve a high level of SA than it is crucial that the system design provides this 
opportunity. There is not a level of SA that is required for each situation but having an 
appropriate level of SA is valuable. The key though is that the information that is 
available needs to be assessed properly so that a solid foundation of SA can be 
developed. Additionally, maintaining SA is a dynamic function that is constantly 
changing, and this requires an individual to dynamically process the ever-changing 
information and process it to the best of their ability at that given time. 

In addition to building SA, an operator’s attention is another key topic within the 
SA realm. If an individual is being attentive, the SA of that individual will typically be 
higher (Endsley & Garland, 2000, p. 9). However, it is important to note that even if an 
individual is being attentive, they may be attending to the wrong cues and their SA could 
be faulty and ultimately result in the committing of errors. 
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According to Endsley and Garland (2000), working memory is an activated subset 
of long-term memory. They suggest, “information from the environment may be 
processed and stored in terms of the activated mental model or schema” (p. 10). The 
concept of a mental model is something that an individual develops in order to visualize 
the information and process it. This concept is believed to be how an individual builds 
SA. These mental models are used within long-term memory and working memory. It is 
this mental model that “is the underlying knowledge that is the basis of SA” (p. 12). 
Again, according to Endsley and Garland, the level of SA depends on how good a mental 
model an individual develops. Applying Endsley and Garland’s three levels of SA can 
develop this mental model. If executed poorly, an individual can develop a poor mental 
model and the results of the situation could be drastic. Parasuranam, Sheridan, and 
Wickens (2008) state that SA is not a choice nor is it general knowledge of material. 
However, SA can be measured by the performance of an individual conducting a task or 
multiple tasks by assessing their choices, and it is these studies that have proven that SA 
is a quantifiable instrument in the construct of cognitive engineering according to the 
authors. 

Stress and workload can be related when analyzing SA. Every person has a 
different capacity for stress, but a level of stress is necessary in order to perform at an 
acceptable level (Endsley, 1995, p. 52). In addition, individuals have different capacities 
for workload. If the workload becomes too much for that individual, the stress will 
increase, and an individual’s SA will begin to decrease (pp. 52-53). This type of human 
factor can impact the SA of an individual, and it is that individual’s capability that 
impacts how stress affects the situation. 

An important aspect of Endsley’s work is the concept of automaticity, which is 
defined as the automatic processing of information with minimal effort. In essence, an 
individual completes required procedures without using too much attention (Endsley, 
1995, p. 45). This is a significant concept because it explains the goal of this prototype. 
Eor a USMC KC-130J aviator executing an lA, the basic procedures will never change, 
yet every lA is different. 
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A key element of the SA of a USMC KC-130J aviator is the use of the aircraft’s 
automation within the cockpit. Over the past few decades, automation has increased 
dramatically. Parasuraman et al. stated that as automation has increased, the trust in the 
automation has increased, and this trust stems from the user’s knowledge of the 
automation and proper manipulation of the automation. However, complacency can cause 
an individual to have too much trusted in the automation, which can have negative 
results. In addition, an individual with not as much knowledge as necessary can rely on 
the automation too much and have negative results as well according to Parasuraman et 
al. It is vital that aviators use the automation efficiently because “working with 
automation have been found to have a diminished ability to detect system errors and 
subsequently perform tasks manually in the face of automation failures as compared with 
manual performance on the same tasks” (Endsley, 1995, p. 53). Conversely, some 
research has found that automation can benefit SA because “automation that reduces 
unnecessary manual work and data integration required to achieve SA may provide 
benefits to both workload and SA” (p. 54). Trust in automation is widely dependent upon 
the individual and what their knowledge is of the necessary material and proper execution 
of the automation (Parasuraman et al., 2008, p. 151). 

C. KNOWLEDGE 

Within the aviation domain, there are key types of learning that occur: procedural 
and conceptual (Dattel, Durso, & Bedard 2009, p. 1964). Procedural “emphasizes the 
step-by-step actions and sequences required to complete a task” while “conceptual 
emphasizes the interrelationship of the system components” (pp. 1964-1965). In 
addition, Cooke, Salas, Cannon-Bowers, and Stout (2000) stated that declarative 
knowledge, which consists of facts, figures, rules, relations and concepts in a task 
domain, is another knowledge (p. 157). Before, conceptual and procedural knowledge is 
explained, declarative knowledge has an overarching impact on the other two types of 
knowledge because it is imperative that an aviator have that type of knowledge 
memorized before evening taking-off. In order to decipher between how conceptual and 
procedural knowledge work within aviation, the two techniques can be applied to 

different aspects of the aviation domain. For example, procedural is beneficial when a 
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typical procedure needs to be followed in order to successfully accomplish a task such as 
normal checklists, communication calls, and landings. If at any time during the situation 
it becomes atypical, conceptual knowledge becomes more beneficial than procedural 
because a “conceptual knowledge of the interrelationships of the system components may 
successfully” remedy the atypical situation (Dattel et ah, 2009, p. 1967). 

Dattel et al. (2009) conducted an experiment that involved separate groups of 
novice aviators learning aviation knowledge such as course rules and landing pattern 
traffic via conceptual and procedural knowledge. At the conclusion of the experiment, 
Dattel et al. determined that the conceptual knowledge technique is better for aviation 
skills. Particularly, Dattel et al. concluded that for non-complex “unfamiliar and atypical 
conditions” conceptual learning is better than procedural (p. 1967). 

D. PART-TASK TRAINING 

Part-task training is an emerging phenomenon that is becoming a crucial 
supplemental resource in many training fields (Stanney, Mourant, and Kennedy 1998, p. 
327). PTTs have offered a more convenient and way of accessing information and 
communication (John, Phillips, Cenydd, Coope, Carleton-Bland, Kamaly-Asl, and Gary, 
2015, p. 156). For example, neurosurgeons in the United Kingdom have developed an 
application for the iPad in order to train neurosurgeon trainees with vital procedures of a 
particular brain surgery in order to reduce a patient’s possible exposure to additional risk 
(p. 155). Initial results have demonstrated promise to the positive effect PTT can have in 
training neurosurgeon trainees with brain surgery procedures. These finding suggest that 
using a PTT as a supplemental resource can have a significant impact on future training 
within many fields. 

Part-task training can provide the user with additional confidence, situation 
awareness, and knowledge to successfully execute important steps within a crucial 
procedure. The Marine Corps has recognized the idea of using modeling and simulation 
in order to maintain the skills of Marines in regards to their duties (Neller, USMC, 2016). 
In particular, the Training and Education Command of the Marine Corps has conducted 
research into Deployable Virtual Training Environment (DVTE) for the purpose of 
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sustaining Marines’ readiness (Brannon & Villandre, 2002, p. 15). It is suggested here 
that the deliberate creation and use of a PTT for training will continue to meet the needs 
of the Marine Corps and could begin to turn the tide of their deployment and 
underutilization for training. 

There are some important human factor elements that need to be examined when 
developing a PTT. Stanney, Mourant, and Kennedy (1998) state that it is important to 
consider how to assess a VE in order to determine the impact on training that has 
occurred through the use of the device. The desire is to have the outcome of the training 
with the VE device be no less than the outcome of its predecessor’s training according to 
the authors. In other words, “the use of a VE should improve task performance when 
transferred to the real-world” (p. 330). Eurthermore, there is also the appeal to have the 
VE device easy to navigate and use (p. 329). In order to accomplish these goals, it is 
recommended that the user experience appropriate levels of immersion and presence 
when using a VE system according to Slate and Wilbur (1997). Presence is defined as a 
“psychological sense of being somewhere else than in one’s immediate physical world” 
while “immersion is a degree to which the technical elements of VE solutions envelop the 
users” (Slater and Wilbur, 1997, pp. 606-607). 

Stanney et al. (1998) suggests that a positive transfer of training can occur as long 
as original learning occurs. In VE, original learning can occur if the immersion and 
presence are representative of the task environment (p. 332). Key aspects of human 
performance that should be considered in the design of the VE device are the “abilities 
and limitations of human sensory and motor physiology” (p. 334). When referring to the 
learning aspect of VEs, “there are strong scientific reasons why the active engagement 
available in VE games rather than passive action of television watching can have better 
retention of learned skills” (p. 344). 

E. SUMMARY 

This chapter described SA, aspects of SA that can impact situations, and how 
factors such as presence, immersion, usability, and performance assessment impact the 
acceptability of the use of VEs for military training. Additionally, the types of knowledge 
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(procedural, declarative, and conceptual) were addressed as well as their relation to 
aviation. Finally, a case is made on how PTT could be used as a viable tool for enhancing 
SA, and the type of learning that is required having a positive transfer of skills and 
knowledge into the intended environment. 
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III. TASK ANALYSIS 


A. INTRODUCTION 

An aviator should have a standard practice for setting-up and performing the 
necessary procedures of an lA. Regardless of what special situation or even emergency 
arises during any particular lA, the aviator with a standard practice for an lA has the 
potential to successfully handle these situations and fly the approach successfully. It is 
the repetition of a standard practice that has a power to permit aviators the ability to 
maintain a high level of SA throughout the approach according to Fowlkes, Dwyer, Oser, 
and Salas (1998). 

The need to understand the tasks that are to be simulated requires an analysis of 
the job. Naval Education and Training Command’s (NETC) Job Duty Task Analysis 
(JDTA) Manual (NETC, 2011) provides a framework in which the task analysis (TA) 
divides a task into separate steps and identifies important characteristics of each step. The 
initial step is to identify the components and sequence of steps of the task. Next, the 
conditions under which the task will be performed must be identified along with any 
triggers or cues that will keep the sequence progressing. Einally, a performance level 
must be administered with each task so that the individual executing the task can be 
evaluated. 

The TA for KC-130J lAPs was conducted in context of SA, knowledge, and PTT 
described. The following section describes the approach and outcomes of the TA that was 
used to create the prototype system. 

B. SITUATION AWARENESS, KNOWLEDGE, AND INSTRUMENT 

APPROACH PROCEDURES 

The KC-130J community within the USMC has a common practice for their 
aviators. Eirst, aviators are to request and receive the weather report at the necessary 
airfield (Department of the Navy, 2014). The weather report is their initial SA builder and 
should be retained throughout the approach. The next step is for the aviator to select the 
correct lA to fly, and an aviator’s conceptual knowledge should assist in choosing the 
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best lA. The next several steps of conducting an lA have many similar aspects that 
require the aviator’s procedural knowledge to be sound. For example, an aviator needs to 
set-up the cockpit instruments properly, have the correct decelerating airspeed, 
descending altitude, and the correct heading. These are just a few of the steps to flying an 
lA, and they often occur at the same time. Rogers, Maurer, Salas, and Fisk, (1997) state 
that it is theorized that tasks, or parts of tasks, can be trained to become habitual or 
automatized. When achieved, automaticity results in the performance of those tasks or 
subtasks with little cognitive effort. An aviator’s automaticity can be developed if the 
same practice is used when conducting an lA and can become paramount in the ability of 
the aviator to maintain a sound SA. 

Throughout the entire lA, an aviator will rely on what is referred to as the three 
separate levels of SA, (1) perception of the elements within the environment; (2) 
comprehension of the current situation; and (3) projection of future status, that Endsley 
discussed (Endsley, 1995, pp. 36-37). While the procedures of an lA should be constant 
for every approach, other elements such as crew relationships, weather, and airfield 
features are different, which project different future status. Successful landings, despite 
the potential problems presented by these variables, are built on a strong foundation of 
conceptual knowledge. These three levels tie into many sublevels such as time, space, 
and decision-making, and those are vital aspects of every lA. 

The SA of an aviator while flying an approach can be impacted by many factors 
such as attention, perception, working memory, and long-term memory. Attention is 
needed in order to properly perceive the information and apply the appropriate memory 
(Endsley, 1995, pp. 40-41). As it is further clarified, working memory conducts the 
active processing of information whereas long-term memory uses stored information to 
counter any limitations of working memory. 

Eor KC-130J aviators, the impact of automation plays an important role in their 
development and maintenance of SA. The aircraft is capable of complete autopilot, which 
consists of navigation, speed control, and altitude control. Endsley points out three 
crucial reasons as to how operators of automated systems may have slower reactions 
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times to problems and longer reeovery times (Endsley, 1995, p. 54). The three reasons 
are: 

a loss of vigilance and increase in complacency associated with the 
assumption of a monitoring role under automation, the difference between 
being an active processor of information in manual processing and a 
passive recipient of information under automation, and a loss of or change 
in the type of feedback provided to operators concerning the state of the 
system under automation (p. 54). 

Considering these circumstances, one can argue that aviators utilize more 
cognitive resources to maintain the required level of SA during automation than when 
manually flying the aircraft because being actively engaged in the task can assist with 
keeping the attention and perception high. 

C. INSTRUMENT APPROACH PROCEDURES 

This section will focus on the prototype questions and cues used as the basis for 
the prototype that was developed for this thesis. These questions and cues were derived 
from the high level TA for an lAP. Additionally, these are color coded to correspond to 
the TA (see Appendix B). For brevity, this section will focus on PF information since the 
PM information is nearly identical. 

1. Prototype Question TA 

Prototype question TA is from the Department of the Navy’s (2014) Naval Air 
Training and Operating Procedures Standardization Flight Manual selects sets of items 
that are part of NATOPS task description have been expressed in the prototype system 
via questions-and-answers as part of the user interface. These questions require the user 
to recognize the correct sequence of steps as presented on the corresponding instrument 
panels. Segments of NATOPS task description that are presented as question-and answer 
form of interface include: 

Step 1. Review Pressurization Panel. 

Step 2. Review Anti-Ice/De-Ice Panel. 

Step 3. Review HUD. 
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Step 4. Review CNBP Menu. 

Step 5. Review AMU Menu. 

Step 6. Review REF/MODE panel. 

Step 7. Review CNI-MU. 

Step 8. Fasten Belts/Harnesses. 

Step 9. Complete Approaeh Brief at appropriate time. 

2. Prototype Visual Cues TA 

Prototype visual eues TA is from the Department of the Navy’s (2014) Naval Air 
Training and Operating Procedures Standardization Flight Manual. There are several 
visual eues that an aviator must reeognize and aet aeeordingly when flying an lA. The 
prototype presents several of those eues by mirroring the aireraft’s layout of the heads up 
display (HUD) on the user interface. Visual cues address the following segments of 
NATOPS task description. These include: 

Select the appropriate NAVAID when cleared by ATC. 

Maintain awareness of the DME. 

Maintain awareness of the altitude. 

Maintain awareness of heading. 

3. Prototype Audio Cues TA 

Prototype audio cues TA is from the Department of the Navy’s (2014) Naval Air 
Training and Operating Procedures Standardization Flight Manual. Throughout an lA, 
radio calls are constant and provide valuable cues that an aviator needs to recognize. The 
scenario that has been presented in our prototype system has a number of common radio 
calls that an aviator should listen to and know what action is expected to happen and 
when to do it. The following list presents elements of NATOPS task description that were 
integrated as audio cues within the prototype. 

Eisten to the weather report to determine which lA to fly. 
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Execute an approach brief once the PM has completed flows. 
Listen to ATC’s clearance for the approach. 

Listen for the mandatory call outs during the approach. 


D. SUMMARY 

This chapter discussed the outcomes of the TA, which was conducted in context 
of SA and requisite knowledge for KC-130J lAs. This TA serves as the basis of the 
prototype and the scenario created to represent an lA for this thesis. 
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IV. DESIGN AND DEVELOPMENT OE THE PROTOTYPE 

TRAINING SYSTEM 


A. INTRODUCTION 

Before a design and development of a prototype system was approached, a survey 
of KC-130J subject matter experts (SMEs) was conducted. The objective of this work 
was to capture information about possible gaps in training, the issues and trends they 
identified with performance of the aviators during FAM stage (with special emphasis on 
lAP), and their suggestions on how a new PTT could fit in and benefit the current 
training regimen. It is important to obtain the current pulse of the fleet while developing 
such a device because it provides a review of the most up-to-date insights and necessities 
as they are seen by the warfighters i.e., the actual users of current and future systems. 

B. SURVEY OF A USER BASE 

The use of human subjects in surveys that includes data collection from those 
individuals and analysis of the same data sets, dictates applying institutional review board 
(IRB) protocol and its adopted requirements and procedure. In the case of this thesis, it 
required permission to proceed with the study from an IRB Committee at NPS and within 
the U.S. Marine Corps; the latter required approval from the survey approval office and 
the human research protection office of the U.S. Marine Corps. (See Appendix C) 

I. Subject Matter Expert Input 

Two KC-130J SME were consulted regarding different issues in this domain - 
their insights and feedback helped us develop the entire survey that would be delivered to 
the target user base. The questions for the SMEs were rather generic and open-ended 
about training lAPs for the KC-130J; their responses allowed us to generate more precise 
questions for the official online survey. The full list of questions submitted to the SMEs 
is in Appendix A; only the most significant responses given by the SMEs are discussed in 
this section. 


21 



The SMEs reported that they observed RPs having problems with basic lAP. One 
SME commented that RPs rely too much on the automation of the KC-130J, which could 
lead to problems escalating before the RP even recognizes the existence of a problem. A 
common theme amongst the SME’s responses was that repetitive exposure to the lAP via 
practice such as chair flying could be very beneficial. Another valuable point commented 
by a SME was that RPs struggle with complex approaches or busy airports. These 
statements were not acquired to validate the purpose of this thesis but rather to confirm 
potential value of repetitive chair flying. The SMEs’ replies confirmed that a PTT could 
be used as a supplemental training tool, as it was outlined in this thesis. That type of 
training solutions can be seen as essential tools for enabling additional training 
opportunities that are needed to maintain SA at an appropriate level. 

2. Online Survey: Procedures and Results 

The EimeSurvey system was used as an instrument that helped generate online 
surveys for RPs and IPs. This system allowed a link to a web-based application be sent to 
potential survey takers, thus eliminating the need for subjects to come to a laboratory and 
fill the physical questionnaire. 

Participation in all surveys was voluntary basis. Unfortunately, only two IPs 
responded to our call, nevertheless valuable information was acquired even with a small 
number of participants. Questions for the RPs addressed several topics that included 
current studying (training) practices, subjects’ perceptions about current lA practices, 
trends in students’ performance, and desired capabilities that future PTT could provide. A 
full list of survey questions is in Appendix A and only the most significant replies of the 
IP survey are summarized in this section. 

The two IP survey results collected in this study reported a common issue 
identified in procedural execution. Procedural execution is not a lack of procedural 
knowledge; in this content it refers to a lack of lAP experience such as performing the 
flows efficiently and accurately. RPs tend to take too much time performing these flows, 
which results in a RP getting well behind the approach’s timeline. Additionally, the IP 
survey reveals that RPs tend to struggle with communications and maintaining SA. Both 
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responses address the importanee of ehair flying. The survey also addressed questions 
pertaining to features desired in a future PTT; that diseussion will be addressed in 
Chapter V. As stated, the RPs’ struggles with flow execution were used as guidance 
during the design phase of the prototype. For example, the in-range flows are one of the 
few items that are visually depicted within the prototype so that a RP can visualize their 
proper sequence. 

C. CONCEPTUAL FRAMEWORK 

The overall conceptual framework for this PTT is very simple. The questions are 
called from a JavaScript Object Notation file, which allows the user to answer the 
questions presented in the scenario. At the end of the scenario execution, the IP can view 
the results of how the user answered the questions from a file once the scenario is 
complete. Figure 1 shows elements of the conceptual framework that has been adopted 
for this PTT. 



Answer 

Questions 


Figure 1. Conceptual framework. 



D. PROGRAMMING AND DEVELOPMENT ENVIRONMENT 

Since this is a prototype training system at a very basic level, many aspects of the 
coding were hard coded, which does not permit any of the scenario to be altered. The 
reasoning behind this concept is based on two primary reasons: 1) it is a proof-of-concept 
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2) the prototype addresses the basic lAP and radio calls a RP should know and expect to 
hear. 


1. Unity Game Engine and C# 

The Unity game engine was used to develop the scenario since it has vast 
capabilities with the most important being that it is compatible with multiple operating 
systems. Unity was primarily developed to assist with the development of video games 
and simulations. In addition, Unity is free to download from the Internet, which makes it 
easily accessible. These characteristics make Unity a viable choice when selecting a 
program to use for an effort such as this thesis. Figure 2 shows a segment of the terrain 
used in the PTT as seen inside the Unity interface. 



Figure 2. Unity game engine. 


2. Images of the PTT Interface 

Figures 3-7 depict different elements of the PTT interface that the user will 
experience during the scenario. The images display different aspects of the scenario 
ranging from the instrument panels with their questions to how the terrain looks at 
different altitudes. 
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Figure 3. Screen shot at 6000 feet. 


Which elements do 
you check? 



Figure 4. Screen shot of instrument panel and question. 
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At what DME must 
you be have all 



Figure 5. Screen shot of approach plate and question. 



Figure 6. Screen shot at 3677 feet. 
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Figure 7. Screen shot of airfield at Twenty-nine Palms 


3. Auditory Cues 

The Audacity application was used to record radio calls, which commonly occur 
during an lA. In order to increase realism within the environment, engine noise was 
added to the scenario. This audio sample was downloaded from a free sound library 
(freesound.com). The use of the Unity development environment allowed for the 
relatively easy addition of the required auditory cures; simple drag and drop action is all 
that is required to add this capability to the final user interface. 

E. DEVELOPMENT 

The initial idea was that this prototype training system would replicate a very 
realistic ability to practice lAP, but through the tasks analysis and SME input, it became 
evident that a system of this complexity was not needed since the goal of this prototype 
was to create a supplemental tool to serve as a PTT. Hence, the scope of the prototype 
was narrowed down to address the procedures and situation awareness associated with 
lA. The user could practice the lA task as often as it was desired, allowing those skills to 
become second nature so that an appropriate level of SA could be maintained while 
executing lAPs. 
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The prototype training system was designed to support the execution of a scenario 
from the beginning of an lA until just before the touchdown upon an airfield. A minimal 
user input is required except for answering a set of questions that test user’s SA and 
knowledge. The prototype depicts the first person view, and it adds an additional visual 
layer that replicates a look and feel of the HUD of the KC-130J (Figure 6). The scenario 
used in the system addresses all the steps described in the NATOPS documentation; the 
same steps must occur for a normal lA along with the standard cockpit and radio calls. 

F. MODES OF USE 

The prototype can be used in three modes or configurations: tablet, laptop, or 
large screen. All three modes of use offer different aspects that can enhance general 
capability of the prototype. The tablet’s advantage is that an individual can practice lAP 
anywhere at any time. A laptop can be used amongst a small group. A large screen device 
can be used in an instructor-led classroom. Figures 8-10 depict the different modes. 



Figure 8. Tablet mode. 
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Figure 9. Laptop mode. 



Figure 10. Large screen mode. 
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G. INFORMAL DEMONSTRATION 

An informal demonstration of the prototype was conducted with the goal to 
acquire valuable feedback that could be applied to future work. 

1. Study Design 

The demonstration population consisted of aviators currently attending the Naval 
Postgraduate School (NPS). A request was submitted to the registrar’s office to acquire a 
list of Navy and Marine aviators so that their presence could be requested for the 
demonstration. Of the thirty-five aviators contacted, eight participated in the 
demonstration. 

For the demonstration, a brief explanation of the thesis and prototype was given. 
Upon completion of the brief, the scenario was shown to the aviators. The scenario lasted 
about eight-and-a-half minutes. At the conclusion of the demonstration, the participants 
handed in written answers to the questionnaire (see Appendix A), thanked for their 
participation, and departed the demonstration area. 

2. Results 

The insights gained during the demonstration coincide with recommendations 
from the IP LimeSurvey: 

• Ability to develop a scenario so that a RP can be evaluated for a 
particular portion of an lA. Currently, the prototype executes an 
approach from start to finish, which does not allow the user to focus on a 
single section of the approach. 

• Provide feedback and a study guide. A suggestion collected in the IP 
survey was that the PTT could provide guidance for the user and suggest 
elements that needed to be improved in order to develop the required lAP 
skills. 

• Ability to study approaches at any airfield. A common comment from 
the demonstration was a desire to have the PTT execute and support lA at 
several airfields with multiple approaches. 

• Ability to select or alter switches/buttons. Both the IP survey results and 
the demonstration expressed a desire to have the PTT provide the user the 
opportunity to select switches or buttons on the instrument panels. Some 
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of the buttons or switches might require a haptic feedback, yet this would 
only need to be a vibration sensation similar to a smart phone’s vibration 
when a button is selected. This type of haptic feedback would permit the 
device to still be transportable. While including haptics could be 
advantageous for the overall learning process, it could also be more than 
what a PTT needs to perform. An analysis of how passive haptics would 
impact a PTT would need to be examined. 

• Voice recognition. The radio calls are prerecorded in this prototype, yet 
many expressed a desire to have a support of voice recognition software so 
that the user could practice radio calls. 

• Moving Screen. The orientation of the viewpoint, first person, in the PTT 
does not change, and a suggestion received from SMEs was to introduce 
the possibility of changing the orientation of the viewpoint independently 
so that a section of the cockpit with the instruments could be seen. 

• Impose time constraint to scenario. For a more advanced PTT, a 
suggestion was to put the user in a time constraint scenario and emulate 
real situations. 

• Pop-up windows. During dead space of the scenario, it has been 
suggested to have pop-up questions or comments to keep the user 
engaged. 

H. SUMMARY 

This chapter discussed different aspects of the design and development of the 
prototype training system. This included a survey of the user base and its results, 
followed by a review of conceptual framework, the user interface that was developed, 
and finally a discussion of the informal demonstration and feedback gathering that was 
done with a help of local SMEs at NPS. 
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V. DISCUSSIONS AND CONCLUSIONS 


A. DISCUSSION 

The increasing accessibility of low-cost, off the shelf technologies presents an 
opportunity to drastically impact how aviation training is accomplished. Recognizing 
this, the thesis presented here looked to answer an overarching research question and two 
additional research questions that support the main question. First, would it be feasible to 
develop a VR system to support training lAPs to improve situation awareness? Next, 
what if any, training gaps exist with the training of lAPs and SA for lAs? Lastly, and 
based on the former, what cues should be included in a PTT to support the practice of 
gaining and maintaining SA during lAs? 

The first question was partially answered. Through a modified instructional 
systems design approach, this thesis demonstrates that a systematic approach to the 
development of a PTT for training lAs could be accomplished in a relatively short 
amount of time in a resource-constrained environment. Unfortunately, due to 
administrative delays, a training effectiveness evaluation was not conducted and the 
second part of our first research question cannot be definitively answered. However, post¬ 
demonstration SME feedback suggests that the prototype developed for this thesis shows 
promise for helping aviators in the KC-130J FRD develop knowledge about lAPs and 
improve their ability to rapidly perceive, assess, and plan their actions during lAs. 

In support of the main question, the exploration of potential training gaps was a 
mandatory step to ensure that the effort of this thesis was warranted. Through this work, 
we discovered that gaps do exist in the form of available opportunities and the fidelity of 
training aides. Further, it is suggested here that a fundamental change in how training is 
considered may be overdue. While technology can be considered merely the vessel in 
which content is delivered, current and future generations of aviators may grow to expect 
particular technologies to be part of their experience. This is not to minimize the value of 
chair flying as a form of mission rehearsal but to offer that aviators using virtual 
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technologies in their everyday lives may have improved training experiences and 
outcomes if these technologies are available for their training. 

Finally, and perhaps most importantly for the development of any training system, 
the identification and incorporation of the system and environmental cues needed to 
support training goals was accomplished. Mistakes at this stage could have disastrous 
results. Ensuring elements such as cockpit displays, environmental conditions, 
communications, time, and space are accurately represented is critical for avoiding 
negative transfer of training for aviators. We are confident that the thorough review of 
available publications, SME interviews, and SME evaluations conducted for this thesis 
enabled us to understand these cues and translate them into the KC-130J lA training 
prototype created. In sum, this thesis was able to achieve the goals as originally stated. 
However, as with many projects such as this, there is still work that must be 
accomplished before a prototype could be released to actually support training. 

B. FUTURE WORK 

The general objective of this thesis was very broad, and future work in this 
domain could accomplish that goal. An entire range of PTTs can become valuable 
training solutions in the future and address a variety of training gaps. Scenarios for other 
missions could be developed; this would permit aviators the repetitive opportunities 
needed to master a plethora of skills required to safely and successfully operate within the 
KC-130J community. In that regard, a very similar concept of PTT could be used for 
other segments of aviation community. An additional line of future work could network 
these PTTs together so that separate communities could conduct joint mission rehearsals 
such as an aerial refueling mission. 

1. Knowledge 

This thesis focused on SA; however, there is a significant amount of work needed 
to analyze the three types of knowledge that were introduced within chapter two. The 
work could focus on how each type is important to an lA and the lAP of the KC-130J in 
support of maintaining SA. The reference for this material within this thesis is a good 
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starting point for knowledge and general aviation, but a focus of knowledge and the KC- 
130J could be beneficial work for the community. 

2. Eye Tracking 

A thesis that could integrate eye tracking system and try to differentiate between a 
novice, junior level, and SME executing an lA. The study could be designed to acquire 
understanding of what each of those three separate groups is looking at during an lA and 
provide valuable feedback on what to do and what not to do during an lA. Eye tracking 
could also provide important insights as to the progression of a trainee through a 
curriculum. If a trainee’s scan pattern repeatedly matches an expert model, it may be 
possible to advance them through a training program more rapidly. Conversely, it would 
also be possible to identify those who may need extra training opportunities or other 
interventions. 

C. CONCLUSIONS 

The importance of developing a PTT that can supplement current training of lAPs 
cannot be understated. The value that a PTT could be easily transported and used at the 
user’s leisure is significant. A PTT can fulfill that gap of repetition and allow an 
individual to master procedural and demonstration knowledge with the ultimate goal of 
developing conceptual knowledge and maintaining sound SA. 
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APPENDIX A. QUESTIONNAIRE AND SURVEYS 


1. SME QUESTIONS 

1. In your experience, what are the types of issues that pilots struggle the most with as 
they begin conducting instrument approach procedures in the actual aircraft? List 
three major issues that, in your view, fit this description. 

2. Given your experience with RPs, how well do you think that the majority of them 
perform instrument approach procedures? 

1 = it is well beyond minimal standards 

2 = it is beyond minimal standards 

3 = it is within minimal standard 

4 = it is below minimal standards 

5 = it is far bellow minimal standards 

3. In your estimate, what is the percentage of RPs that underperform and what is the 
percentage or RPs that excel in instrument approach procedures? 

4. What is your recommendation when you notice a RP is having problems with 
instrument approach procedures - what training approach do you recommend a RP 
take in terms of the skill remediation? 

5. Aside from the answers provided from the previous question, list any additional 
information or comments that relates to trends associated with instrument approach 
procedures. 

6. What supplemental instructional techniques (ones that are not directly required by 
the Training & Readiness (T&R) manual) are used in your squadron in order to 
better prepare RPs for instrument approach procedures? 

7. In your opinion, what is the most effective supplemental technique for preparing 
RPs for instrument approach procedures? 

8. What element of that supplemental technique do you think is very effective for 
preparing RPs for instrument approach procedures? 

9. In your opinion, how often should RPs ideally use that type of supplemental 
technique? 

10. In your opinion, is there any additional training solution that should be made 
available to the RPs to better prepare them for instrument approach procedures in 
the actual aircraft? 

11. What major learning and training objectives should that system support? 
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CONSENT 


Naval Postgraduate School 
Consent to Participate in Research 

AUTHORITY: 10 U.S.C. 5013; 10 U.S.C. 5041; 10 U.S.C. 1074f; 32 CFR 64.4; DoD Instruction 1215.13; DoD Instruction 3001.02; aCSM 
3150.13C; DoD Instruction 6490.03; SECNAVINST 1770.3D; MCO 7220.50B; and E.O. 9397 (SSN), as amended. 

Introduction. You are invited to participate in a research study entitled Proof-of-concept Part-task Trainer to Enhance Situational 
Awareness for Instrument Approach Procedures in Aviation Domain. The purpose of this research study is to investigate important 
issues novice pilots have flying instrument approach procedures. 

Procedures. You will be asked to answer several questions utilizing an online survey tool called LimeSurvey. The survey is focused on 
your experiences and understandings of current instrument approach procedure techniques. The surveys should take approximately 30 
minutes to complete. 

Location. The survey will be conducted online. 

Cost. There is no cost to participate in this research study. 

Voluntary Nature of the Study. Your participation in this study is strictly voluntary. It is important to know that if you choose to 
participate you can change your mind at any time and withdraw from the study. You will not be penalized in any way or lose any 
benefits to which you would otherwise be entitled if you choose not to participate in this study or to withdraw. The alternative to 
participating in the research is to not participate in the research. 

Potential Risks and Discomforts. There is a minimal risk of breach of confidentiality and the survey process does not involve greater 
than minimal risk. 

Anticipated Benefits. Anticipated benefits from this study are for the Marine Corps and any other services that conduct instrument 
approach procedures (lAP). Data collected will help In identifying methods to effectively use virtual environments for the conduct of a 
part task trainer within the lAP mission set. You will not directly benefit from your participation in this research. 

Compensation for Participation. No tangible compensation will be given. 

Confidentiality & Privacy Act. Any information that is obtained during this study will be kept confidential to the full extent permitted by 
law. Information collected will be removed from LimeSurvey and stored by the Principle Investigator. All efforts, within reason, will be 
made to keep your personal information in your research record confidential, but total confidentiality cannot be guaranteed. No 
information will be publicly accessible that could identify you as a participant. Understand that records of your participation will be 
maintained by NPS for ten years. 

Points of Contact. If you have any questions or comments about the research, or you experience an injury or have questions about any 
discomforts that you experience while taking part in this study please contact the Principal Investigator, Dr. Amela Sadagic, (831) 656- 
3819, asadagic@nps.edu. Questions about your rights as a research subject or any other concerns may be addressed to the Navy 
Postgraduate School IRB Chair, Dr. Larry Shattuck, lgshattu@nps.edu. 

Statement of Consent. I have read the information provided above. I have been given the opportunity to ask questions and all the 
questions have been answered to my satisfaction. 1 have been provided a copy of this form for my records and I agree to participate in 
this study. I understand that by agreeing to participate in this research and signing this form, I do not waive any of my legal rights. 

1 consent to participate in the research. 


Yes No 
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IP 


IP Survey 


1. Does a training gap exist between the classroom/computer'based trainers and the simulator? 
Choose one of the following answers 

Yes Please enter your comment here: 

No 


2. Does a training gap exist between the simulator and the aircraft during the familiarization (FAM) stage? 
Choose one of the following answers 

Yes Please enter your comment here: _ 

No 


3. During the first flight event in the PAM stage, most replacement pilots (RPs) are able to conduct all procedures and communications 
of instrument approach (lA) at an acceptable standard per the grade sheet. 

Choose one of the following answers 

Strongly Agree 
Agree 

Somewhat Agree 
Neither Agree nor Disagree 
Somewhat Disagree 
Disagree 

Strongly Disagree 


4. In your opinion, what are the top three difficulties the RPs have during FAM flights? (e.g. - Flows, Checklists, Getting behind the 
aircra^ comms, etc.) 


5. List any additional information or comment that relates to trends associated with the performance of RPs in FAM stage. 
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6. Which of the following supplemental instructional techniques that are not directly required by the Training & Readiness (T&R) 
manual, are used in the squadron in order to better prepare RPs for the FAM stage? Check any that applies, and select frequency with 
which it is used. 


Daily Weekly Monthly 

Chair Rying 
Formal quizzes/tests 

Self study of supplemental 
readings 


7. In your opinion, what is the effectiveness of each supplemental technique used to prepare RPs for FAM flights? 

Neither 

Very Somewhat Effective nor Somewhat 

Effective Effective Effective Ineffective Ineffective Ineffective 

Chair Plying 
Formal Quizzes/tests 

Self Study of Supplemental 
Readings 


8. In your opinion, how often should RPs ideally participate in chair flying sessions while preparing for the FAM stage? 
Choose one of the following answers 

Daily 

Weekly 

Monthly 

Other: 

9. List three most significant reasons why you recommend using chair flying to RPs under your instruction: 


10. In your opinion, what are the limitations of chair flying? 

Check any that apply 

Lack of IP availability to lead evolution. 

Lack of IP availability to create new scenarios/scripts. 

Time constraints for all participating aircrew (Crew Day, Crew Rest, etc). 
The inability to train large groups at one time. 

Other: 


Rarely 


Very 

Ineffective 
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11. RPs who perform well on format quizzes/tests more likely to perform well in the aircraft during FAM flights? 
Choose one of the following answers 

Strongly Agree 
Agree 

Somewhat Agree 
Neither Agree nor Disagree 
Somewhat Disagree 
Disagree 

Strongly Disagree 

12. List the three most significant reasons why you think RPs should take formal quizzes/tests. 


1 . 

2 . 

3 . 


13. Describe other supplemental Instructional techniques that you use or think those could help RPs become better prepared to conduct 
their first lAPs. This may include things that RPs do on there own time without direct involvement of an IP. 

1 . 

2 . 

3 . 


14. Imagine you are tasked with designing a new tool to help prepare RPs for their first flight involving lAPs in a flight simulator. The 
system could, for example, allow visualization of lAP and practicing of a specific skill set associated with instrument approach. What 
major training objectives should that system support? 


1 . 

2 . 

3 . 


15. The features of that system may allow an IP to define details of some scenario and a set of popup questions that could be presented 
to a RP at various checkpoints or phases of the flight to quiz the RP's knowledge. What other system capabilities would you recommend 
that such training system could have? 

Check any that apply 

Ability to question the knowledge about in-range and approach checklists 

Ability to have text or voice operated radio calls 

Ability to select from a set of prebuilt airfields with their approaches 

Other: 


16. In order to test RP's situational awareness in the training system described in previous question, what element of RP's situational 
awareness would you test and how would you go about it in that training system? 


17. What would be the best hardware platform to support desired functionality of this training system? 
Choose one of the following answers 

Computer with wall size projection display 
Computer with large monitor 
Laptop Computer 
Tablet with touch screen 
Smart Phone 

18. What instructor billets do you currently hold In the squadron? 


1 . 

2 . 

3 . 

4 . 

5 . 

19. How many years of experience do you have as a flight instructor? 
Only numbers may be entered in this field. 
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RP 


RP Survey 


1. Have you conducted your first flight event In the familiarization (FAM) stage of the Training and Readiness manual, and if you did, 
when did that occur? Please enter date In the comment box. 

Choose one of the following answers 

Yes Please enter your comment here; _ 

No 


2. In your opinion, do you have enough opportunities to practice instrument approaches (lAPs) to stay proficient? 

Yes No 

3. During the first flight event in the FAM stage, X was able to conduct all lAP procedures and communications at an acceptable standard 
per the grade sheet. 

Choose one of the following answers 

Strongly Agree 
Agree 

Somewhat Agree 
Neither Agree nor Disagree 
Somewhat Disagree 
Disagree 

Strongly Disagree 


4. In your opinion, what are the top three difficulties you had during these flights? (e.g. • Flows, Checklists, Getting behind the aircraft, 
comms, etc.) 


1 . 


3 . 


5. List any additional information or comments that relates to trends associated with your performance during FAM stage. 


1 . 

2 . 

3 . 
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6. Which of the following supplemental instructional techniques that are not directly required by the Training & Readiness (T&R) 
manual, are used in the squadron in order to better prepare you for the FAM stage? Check any that applies, and select frequency with 
which it is used. 


Daily 


Weekly 


Monthly 


Chair Flying 
Formal guizzes/tests 

Self study of supplemental 
readings 


7. In your opinion, what is the effectiveness of each supplemental technique used to prepare you for FAM flights? 


Neither 

Very Somewhat Effective nor Somewhat 

Effective Effective Effective Ineffective Ineffective Ineffective 

Chair Flying 
Formal Quizzes/tests 

Self Study of supplemental 
Readings 

Other 


8. In your opinion, how often should you ideally participate in chair flying sessions while preparing for the FAM stage? 
Choose one of the following answers 

Daily 

Weekly 

Monthly 

Other: 

9. List the most significant reasons why you use chair flying. 


1 . 

2 . 

3 . 

10. In your opinion, what are the limitations of chair flying? Check all that apply: 
Check any that apply 

Lack of IP availability to lead evolution 

Lack of IP availability to create new scenarios/scripts 

Time constraints for all participating aircrew (Crew Day, Crew Rest, etc) 

The inability to train large groups at one time 
Other: 


Very 

Ineffective 
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11. RPs who perform well on formal quizzes/tests more likely to perform well in the aircraft during FAM flights? 
Choose one of the following answers 

Strongly Agree 
Agree 

Somewhat Agree 
Neither Agree nor Disagree 
Somewhat Disagree 
Disagree 

Strongly Disagree 

12. List three significant reasons why you think RPs should take formal quizzes/tests. 

1 . 

2 . 

3 . 


13. Describe other supplemental instructional techniques that you use or think those could help RPs become better prepared to conduct 
their first lAPs. This may Include things you do on your own time without direct involvement of an IP. 


14. Imagine you are tasked with designing a new tool to help prepare RPs for their first flight involving lAPs in a flight simulator. The 
system could, for example, allow visualization of lAP and practicing of a specific skill set associated with instrument approach. What 
major training objectives should that system support? 


1 . 

2 . 


3 . 


15. The system may allow one to define details of some scenario and a set of popup questions that could be presented to a RP at various 
checkpoints or phases of the fight to quiz the RP's knowledge. What other system capabilities would you recommend that such training 
system could have? 

Check any that apply 

Ability to question the knowledge about in-range and approach checklists 

Ability to have text or voice operated radio calls 

Ability to select from a set of prebuilt airfields with their approaches 

Other: 


5. DEMONSTRATION QUESTIONS 

What functionality is currently missing from the prototype? 

Which functionality of demoed prototype worked? 

Which functionality of demoed prototype did not work? What was missing? 
Which functionality of demoed prototype should be refined (corrected)? 
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APPENDIX B: TASK ANALYSIS 


1. NATOPS TASK DECOMPOSITION 

TA is from the Department of the Navy’s (2014) Naval Air Training and Operating 
Procedures Standardization Flight Manual. 

a. PF In-Range Flows 



Figure 11. PF In-Range Flows 


1. PRESSURIZATION — Set. 

a. Pressure Mode Select — Set. 

b. Rate Selector — Set. 

c. LND/CONST — Set. 

d. PM states the settings of the pressure mode select switch, rate selector knob and 

LDG/CONST altitude. 

2. Anti-Ice/De-Ice — As required. 

a. WING/EMP switch ON, ANTI-ICE (as required, if icing is anticipated). 

CAUTION 

The wing/empennage anti-ice/de-ice system should not be operated in the anti-icing 
mode except during approach and landing. Operation below -30 degrees Celsius at power 
settings above 2,500 HP, or above 20,000 feet pressure altitude at any power setting may 
degrade engine life. 
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Note 

• Engine anti-ice should be ON if SAT is less than or equal to 10 degrees Celsius 
with visible moisture. 

• Wing/empennage anti-icing in anti-ice mode should be turned on in sufficient 
time prior to landing to ensure the vertical stabilizer is completely anti-iced. This 
may take as long as 2.5 minutes. Failure to do so will cause the X-WIND 
LIMITED ACAWS message to be displayed when the landing gear are lowered, 
and the side slip warning system will limit crosswind landing capability if icing is 
detected. 

• Failure to turn on wing/empennage anti-icing in anti-ice mode prior to lowering 
the landing gear when icing is detected causes the WING/EMP A/I NOT ON 
ACAWS message to be displayed. 

3. HUD — Set as necessary for the approach. 

4. CNBP — Set. 

a. NAVAIDs. 

(1) Set for required approach NAVAID guidance — (as required). 

(2) NAVAIDs — Set as required. 

Note 

• Ensure that NAVAIDs are tuned, identified and selected and that the instruments 
and radar are set for the approach. 

• The standard NAVAID Tuning procedure is to use the station identification 
letters. After tuning the station, verify the frequency. 

5. AMU — Set. 

a. HDDs — Set to desired approach configuration. 

b. NAV SELECT. 

(1) CDI SOURCE — Select the approach NAVAID. 

(2) PNTR I/PNTR 2 — Set as required. 

(3) COURSE knob — Set. 

(4) CDI SOURCE — Set. 

Note 

• Set the planned final approach course if able. 

• If the autopilot is engaged on the INAV controlling solution side and the 

• CDI source is changed to set up for the approach, the NAV mode will disengage. 
Once the course is set for the VOR or TACAN approach, the CDI source can be 
returned to the INAV solution. Ensure NAV mode is reengaged or heading mode 
is selected if INAV is reselected as the CDI source. 

• If changing CDI source in order to set the course for an approach, ensure the CDI 
source is reset to INAV and the flight director is reengaged if applicable. 
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c. NAV RADAR — Select overlays. 

d. Digital Map Display — As required. 

e. EMS EVENT RECORD — Press. 

Note 

EMS EVENT RECORD can be pressed anytime level flight and constant airspeed can be 
maintained for 30 seconds. EMS EVENT RECORD only needs to be pressed once per 
flight to record propeller balance data. 

6. REE/MODE panel — Set. 

a. RAD AET — Set. 

(1) Precision Approach — 50 feet below HAT. 

(2) Non-Precision Approach — 250 feet. 

(3) Circling/Visual Approach — 300 feet. 

(4) VPR Pattern/Tactical — Set (As required). 

b. EPA — Set. 

c. MINS — Set. 

(1) lER Approach — Set published minimums. 

(2) VPR Approach — Set (As required). 

d. BARO SET — Set. 

7. CNI-MU — Set. 

a. ROUTE/EEGS 

WARNING 

Terminal area waypoints and fixes shall be identified by reference to the appropriate 
navigational aid as published in the approved approach procedure. 

Note 

Approach construction in the CNI should be considered as a secondary priority to the 
accurate programming of flow elements that precede it. If time permits, construct the 
approach in the CNI-MU to represent the approach as closely as possible. 

b. TOED INIT. 

(1) TOED INIT entries. 

(2) Check Eanding Speeds. 

(3) Check Eanding Distances. 

CAUTION 

Ensure the cargo weight used by the CNI-MU for landing data is current. If an airdrop 
has been performed or cargo jettisoned, input the new weight so that the system can 
calculate accurate landing data. 


Note 

If landing data is manually calculated, new landing data will be required for aircraft 
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weight change greater than 5,000 pounds, pressure altitude change greater than 1,000 feet 
or outside air temperature change greater than 5 degrees Celsius. 

c. MC INDEX. 

(1) V SPEEDS. 

(2) TACTICAE PEOT. 

d. INDEX. 

8. Belts/Harnesses — Eastened. 

9. Approach Briefing — Complete. 

a. Reference Chapter 16 for the arrival-briefing guide. 

2. Approach Brief Example 

“We will fly radar vectors to the lES RWY 17 at Navy Eort Worth. The weather is 800 
with 2 miles visibility, the required weather is 200 and 3/4. The field elevation is 650, 
650 is set in the pressurization panel. The NAVAIDs are set with I-NEW in the VORs 
and NEW in the TACANs. I will fly the approach with VORl and the CDI course is set 
to 174. DH is 836, 840 is set in MINS. RAD AET is set 150. Speeds for a 100% flap 
landing are 135, 125, 118. There is an EZ in the CNI. Stable approach gate is 1650.1 will 
fly automated with autothrottles. There is no significant terrain, but there are numerous 
towers along the approach corridor. The published missed approach instructions are 
climb heading 174 to 3000, intercept the EUZ 218 radial to MOEKE and hold.” 

3. IN-RANGE CHECKLIST 

Trigger: At the completion of the Crew Briefing the PE should call for the checklist. 

1. Altimeters: “Set_” (CP). 

“Set_twice” (P). 

WARNING 

Altimeters will be set to station pressure (QNH/QEE) if available when transiting the 
transition level. Altimeters may be set when above, but cleared through, the transition 
level. The In-Range Checklist shall not be called complete until the QNH/QEE has been 
set. 


Note 

The standby altimeter shall be set separately. 

2. Pressurization: “Set,_,_,_” (PM). 

a. PM states the settings of the pressure mode select switch, rate selector knob and 
EDG/CONST altitude. 

3. ANTI-ICE/DE-ICE switch: “_” (PM). 

a. PM states the setting. 
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4. ACAWS: “Checked” (PM). 

5. Lights: “Set” (PM). 


Note 

Consideration should be given to using wing tip taxi lights, formation lights and pod 
illumination lights when descending into high-density traffic areas. 

6. TCAS: “_,_” (PM). 

a. TA/RA — As required. 

b. ABV/Below/Normal — Check. 

c. PM states TCAS mode setting. 

7. Belts/Harnesses: “Fastened” (PM). 

8. Passengers and Cargo: “Secured” (LM/CC/CM). 

9. In-Range Checklist: “Complete” (PM). 

4. APPROACH CHECKLIST 

Trigger: When the final approach navaid is set in the CDI source of the AMU NAV 
SELECT page. 


Note 

The APPROACH CHECKLIST should only be used when: 

• NAVAIDs have been tuned and identified, if applicable, and 

• Below TA/TL, and 

• Entering the terminal environment. 

1. Altimeters: “Set_” (CP). 

“Set_twice” (P). 

a. Whenever the altimeters are reset, both pilots state the new setting. 

2. NAVAIDs: “Set_” (PM). 

a. The PM states the primary approach NAVAI D Erequency. 

3. CDI Source: “_,_” (PM, PE). 

a. CDI source set for approach NAVAID. 

b. Course Select Knob — Set. 

c. Both pilots state the CDI source selected and CDI course. 

4. Rad Alt: “_Set” (PM). 

a. The PM states the radar altimeter set in the REE/SET Mode Select panel. 
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5. Minimums: “_Set” (PM). 

a. The PM states the approach minimums set in the REF/SET Mode Select panel. 

6. FUEE MANAGEMENT panel: “_” (PM/ACM). 

a. The PM or the ACM turns the SPR DRAIN switch ON, if required, and verifies 
the fuel panel configuration. 

b. Fuel quantity — Check. 

c. Distribution — Check. 

d. State the fuel quantity, sink rate limit, and SPR drained. 

Note 

• Do not transfer fuel from tank to tank during landings or touch and go landings. 

• Refer to Fuel Management, Approach and Eanding procedures for allowable fuel 
system configuration. 

7. Briefing: “Complete” (PE). 


Note 

Refer to Chapter 16 for the Arrival Briefing. 

8. Approach Checklist: “Complete” (PM). 

5. BEFORE LANDING FLOW 

The pilot may direct lowering of the flaps and gear before calling for the BEFORE 
EANDING Checklist. 


Note 

• Wing/empennage anti-icing in anti-ice mode should be turned on in sufficient 
time prior to landing to ensure the vertical stabilizer is completely anti-iced. This 
may take as long as 2.5 minutes. Failure to do so will cause the X-WIND 
EIMITED ACAWS message to be displayed when the landing gear are lowered, 
and the side slip warning system will limit crosswind landing capability if icing is 
detected. 

• Failure to turn on wing/empennage anti-icing in anti-ice mode prior to lowering 
the landing gear when icing is detected causes the WING/EMP A/I NOT ON 
ACAWS message to be displayed. 

1. FEAPS: “FEAPS 50” (PE). 


CAUTION 

Changing the direction of flaps travel while the flaps are moving may damage the flaps 
motor. 


a. The PM selects the flap handle to 50% and reports when the flaps are at 50%. 
“Flaps are 50” (PM). 
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Note 

Normal landing configuration is with 100 percent flaps and may be selected after the gear 
is reported down and prior to crossing the threshold. 

2. Gear: “Gear Down Before Landing Checklist” (PF). 

3. Lights: Extended and On (PM). 

a. Landing lights — Extended and On (As required). 

b. TAXI EIGHTS switch — ON. 

c. WINGTIP TAXI EIGHTS switch — As required. 

CAUTION 

Extending landing lights while illuminated may induce pilot vertigo when in IMG 
conditions. 


Note 

The Pilot Monitoring should operate the gear handle. 

6. BEFORE LANDING CHECKLIST 

1. FLAPS: “FLAPS 50” (PM). 

2. GEAR: “3 Down” (PM). 

“3 Down, Centered” (PF). 


CAUTION 

Check the nosewheel steering indicator to ensure the nosewheel is centered. 

Note 

Ensure anti-skid on and no anti-skid ACAWS messages displayed. 

3. Lights: “_” (PM). 

a. PM states whether the landing lights are extended or retracted and On or Off. 

4. Before Landing Checklist: “Complete” (PM). 
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APPENDIX C: RESEARCH APPROVALS 


1. NPSIRB 



Naval Postgradua-te School 

Human Research Protection Program 


From: President, Naval Postgraduate School (NPS) 

To: Dr. Amela Sadagic, Modeling, Virtual Environments and 

Simulation Institute (MOVES) 

LCDR Lee Sciarini, USN 
Capt Nicholas Arthur, USMC 
Via: Vice Chair, Institutional Review Board (IRB) 

Subj: PROOF-OF-CONCEPT PART-TASK TRAINER TO ENHANCE SITUATIONAL 

AWARENESS FOR INSTRUMENT APPROACH PROCEDURES IN AVIATION 
DOMAIN 

Enel: (1) Approved IRB Protocol Amendment 

1. The NPS IRB is pleased to inform you that the NPS President 
has approved your protocol amendment (NPS IRB# NPS.2017.0029- 
AM01-EP7-A). The approved IRB Protocol is found in enclosure 
(1). Completion of the CITI Research Ethics Training has been 
confirmed. 

2. This approval expires on 5 March 2018. If additional time 
is required to complete the research, a continuing review report 
must be approved by the IRB and NPS President prior to the 
expiration of approval. At expiration all research (subject 
recruitment, data collection, analysis of data containing PII) 
must cease. 

3. You are required to obtain consent according to the 
procedure provided in the approved protocol. 

4. You are required to report to the IRB any unanticipated 
problems or serious adverse events to the NPS IRB within 24 
hours of the occurrence. 

5. Any proposed changes in IRB approved research must be 
reviewed and approved by the NPS IRB and NPS President prior to 
implementation except where necessary to eliminate apparent 
immediate hazards to research participants and subjects. 

6. As the Principal Investigator (PI) it is your responsibility 
to ensure that the research and the actions of all project 
personnel involved in conducting this study will conform with 
the IRB approved protocol and IRB requirements/policies. 
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7. At completion of the research, no later than expiration of 
approval, the PI will close the protocol by submitting an End of 
Experiment Report. 


K. J. Euske 


C«i*« 

OM i 

Xm C*US 

ft— MiytnTi Manit.amr 


Kenneth J. Euske, PhD 
Vice Chair 

Institutional Review Board 




Ronald A. Route 

Vice Admiral, U.S. Navy (Ret.) 

President, Naval Postgraduate School 


Date 


MAR 3 0 2017 
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2 , 


DEPUTY COMMANDANT OF AVIATION 


DEPARTMENT OF THE NAVY 
HEADQUARTERS UNITED STATES HARINE CORPS 
3000 MARINE CORPS PENTAGON 
WASHINGTON, DC 20350-3000 

iHittn.vMrc«io 

3970 
AVN 

6 Apr 17 

From; Deputy Commandant for Aviation 

To: Captain Nicholas L. Arthur 1285751540/7557 USMC 

Subj: RESEARCH SPONSORSHIP FOR CAPTAIN NICHOLAS L. ARTHUR 

1285751540/7557 USMC 

1. I support the research proposed by Captain Nicholas Arthur 
regarding the "Proof-of-concept Part-task Trainer to Enhance 
Situational Awareness for Instrument Approach Procedures in Aviation 
Domain" within the broader research area of Modeling and Simulations. 

2. The part-task trainer can be used as a bridge between basic chair 
flying and actually flying approaches in a full-motion simulator or 
even the aircraft. This proof of concept will increase the likelihood 
that the ppot will be capable of successfully flying the Approach 
while conducting additional tasks that often occur. 

3. The research will focus on the following questions: 

a. What is the feasibility of developing a prototype virtual 
reality (VR) system that supports the training of instrument approach 
procedures and situational awareness? 

b. Are there training gaps identified in connection with 
procedures and situational awareness needed for conducting an 
instrument approach by the pilots? 

c. On what cues (cockpit, environmental, CRM, ATC, haptic, etc.) 
should KC-130J FRD students focus in support of maintaining 
situational awareness during instrument approaches? 

4. This research will support the use of modeling and simulations for 
the Aviation community and provide cost effective training 
enhancements for pilots. 

5. Colonel Todd Lyons, Associate Dean of Research at the Naval 
Postgraduate School, can address research specific topics at 
831-884-8317 or twly 0 n 3 @nps.edu . 



J. M. DAVIS 
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3. 


USMC HUMAN RESEARCH PROTECTION 



DEPARTMENT OF THE NAVY 

HEADQUARTERS UNITED STATES MARINE CORPS 
3300 RUSSELL ROAD 
QUANTICO, VA 22134-5001 

3900 
C4623 
7 Apr 17 


From: Human Research Protection Official, U.S. Marine Corps (Attention: Ms Leah 

Watson), 1019 Elliot Rd, Quantico, VA 22134 

To: Dr. Amelia Sadagic, Modeling, Virtual Environments and Simulation 

Institute (MOVES), Eaculty Advisor 

Eieutenant Commander Eee Sciarini, USN, Eaculty Advisor, Naval Post 
Graduate School, Monterey, CA 

Captain Nicholas Arthur, USMC, Student Investigator, Naval Post 
Graduate School, Monterey, CA 

Subj: HUMAN RESEARCH PROTECTION PROGRAM U.S. MARINE CORPS 

ADMINISTRATIVE 17 REVIEW OE PROPOSED STUDY: “PROOE-OE- 
CONCEPT PART-TASK TRAINER TO ENHANCE STITUATIONAE 
AWARENESS EOR INSTRUMENT APPROACH PROCEDURES IN 
AVIATION DOMAIN” 

Ref: (a) DoDI 3216.02 

(b) SECNAVINST 3900.16D 

(c) MCO 3900.18 

(d) U.S. Marine Corps Human Research Protection Program (HRPP) Policy 
and Procedures (27 Sep 16) 

(e) DOD N-40042 Naval Postgraduate School DOD-Navy Assurance 

(f) MCO 5300.18 Marine Corps Survey Program 

Enel: (1) NPS.2017.0029-IR-EP7-A 

(2) NPS.2017.0029-AM01-EP7-A 

(3) DC Aviation Itr 3970 AVN of 6 Apr 7 

1. Per references (a) through (d), I have performed an Administrative Review, on behalf 
of the Marine Corps, of the research titled “Proof-of-Concept Part-Task Trainer to 
Enhance Situational Awareness for Instrument Approach Procedures in Aviation 
Domain.” The objective of this research is to explore the feasibility of developing a 
prototype virtual reality (VR) system that supports the training of instrument approach 
procedures and situational awareness. This research is being conducted as partial 
fulfillment of academic requirements for Captain Arthur as a student at the Naval 
Postgraduate School (NPS). 


56 



2. The research plan at enclosure (1) and the first amendment at enclosure (2), as 
submitted to the NFS IRB, has been reviewed by this office to ensure compliance with 
requirements the U.S. Marine Corps Human Research Protection Program. 


3. As outlined in the research plan at enclosure (1), the research will involve voluntary 
subject participation in an online survey about their understanding of current instrument 
approach procedure training techniques. The results will be used by the researcher in 
determining the prototype interface design. Enclosure (1) also includes the Commanding 
Officer approval for solicitation of subjects, from the Commanding Officers of VMGR- 
152, VMGR-252, and VMGR-352. Enclosure (3), from the Deputy Commandant for 
Aviation, provides the General Officer level support required for research being 
conducted by institutions other than the Marine Corps. 

4. Reference (e) is the DOD Institutional Assurance for Naval Postgraduate School which 
expires 30 November 2019. As reflected at enclosures (1), the 

NPS IRB reviewed and approved your research proposal as meeting criteria for 
Expedited Review under Categories 7 and approved waiver of the requirement for 
documentation of informed consent under 32 CER 219.117((2). 

5. Review by the NPS IRB and the DC Aviation approval satisfy Marine Corps review 
requirements for this study. Approval by the NPS IRB assures that the members of the 
research team have completed the required research ethics training. You are required to 
inform this office, as well as your approving IRB, if there are any changes or 
amendments to your protocol. No further review or approval from this office is required. 
It is noted that the NPS IRB approval for this research expires 5 March 2018. A copy of 
the NPS IRB’s approval of any continuing review for this research beyond the current 
expiration date must be submitted to this office. 

6. Per reference (f) review and approval by the USMC Survey Office is required prior to 
execution of this survey. 

7. If you have any questions or require further information, please don’t hesitate to 
contact me at (703) 432-2566, email leah.watson@usmc.mil. I wish you success with 
your study and appreciate your patience in complying with our review and approval 
process. 


E. B. WATSON 

Chair, Institutional Review Board 


Copy to: 

USMC Survey Office 
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4. 


USMC SURVEY OFFICE 


On Apr 17, 2017, at 11:47 AM, Hogan Maj Kerry A 
< kerrv.hoaan@usmc.mil > wrote: 

Good Afternoon, 

The DC, M&RA has approved your survey. Your SCN is: USMC-SR- 
17008. 

You are cleared to launch the survey. 

R/S 

Maj Hogan 
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